Learning Kernel-Based Halfspaces with the Zero-One Loss
نویسندگان
چکیده
We describe and analyze a new algorithm for agnostically learning kernel-based halfspaces with respect to the zero-one loss function. Unlike most previous formulations which rely on surrogate convex loss functions (e.g. hinge-loss in SVM and log-loss in logistic regression), we provide finite time/sample guarantees with respect to the more natural zero-one loss function. The proposed algorithm can learn kernel-based halfspaces in worst-case time poly(exp(L log(L/ ))), for any distribution, where L is a Lipschitz constant (which can be thought of as the reciprocal of the margin), and the learned classifier is worse than the optimal halfspace by at most . We also prove a hardness result, showing that under a certain cryptographic assumption, no algorithm can learn kernel-based halfspaces in time polynomial in L.
منابع مشابه
Learning Kernel-Based Halfspaces with the 0-1 Loss
We describe and analyze a new algorithm for agnostically learning kernel-based halfspaces with respect to the 0-1 loss function. Unlike most of the previous formulations, which rely on surrogate convex loss functions (e.g., hinge-loss in support vector machines (SVMs) and log-loss in logistic regression), we provide finite time/sample guarantees with respect to the more natural 0-1 loss functio...
متن کاملLearning Intersections of Halfspaces with a Margin
We give a new algorithm for learning intersections of halfspaces with a margin, i.e. under the assumption that no example lies too close to any separating hyperplane. Our algorithm combines random projection techniques for dimensionality reduction, polynomial threshold function constructions, and kernel methods. The algorithm is fast and simple. It learns a broader class of functions and achiev...
متن کاملKernels from Probabilistic Models for Multiclass Classification and Reranking with Arbitrary Loss Function
Many classification problems involve loss functions different from the usual zero-one classification loss. In recent years, several approaches to accommodate loss functions in kernel-based learning algorithms have been suggested, but the construction of kernels has not been motivated by specific loss functions. We propose a method for deriving kernels from probabilistic models, which is tailore...
متن کاملMoment-Matching Polynomials
We give a new framework for proving the existence of low-degree, polynomial approximators for Boolean functions with respect to broad classes of non-product distributions. Our proofs use techniques related to the classical moment problem and deviate significantly from known Fourier-based methods, which require the underlying distribution to have some product structure. Our main application is t...
متن کاملAgnostically Learning Halfspaces with Margin Errors
We describe and analyze a new algorithm for agnostically learning half-spaces with respect to the margin error rate. Roughly speaking, this corre-sponds to the worst-case error rate after each point is perturbed by a noisevector of length at most μ. Margin based analysis is widely used in learningtheory and is considered the most successful theoretical explanation for thesta...
متن کامل